The continuous spread of SARS-CoV-2 calls for more direct-acting antiviral agents to combat the highly infectious variants. The main protease (Mpro) is an promising target for anti-SARS-CoV-2 drug design. Here, we report the discovery of potent non-covalent non-peptide Mpro inhibitors featuring a 1,2,4-trisubstituted piperazine scaffold. We systematically modified the non-covalent hit MCULE-5948770040 by structure-based rational design combined with multi-site binding and privileged structure assembly strategies. The optimized compound GC-14 inhibits Mpro with high potency (IC50 = 0.40 μM) and displays excellent antiviral activity (EC50 = 1.1 μM), being more potent than Remdesivir. Notably, GC-14 exhibits low cytotoxicity (CC50 > 100 μM) and excellent target selectivity for SARS-CoV-2 Mpro (IC50 > 50 μM for cathepsins B, F, K, L, and caspase 3). X-ray co-crystal structures prove that the inhibitors occupy multiple subpockets by critical non-covalent interactions. These studies may provide a basis for developing a more efficient and safer therapy for COVID-19.